Managing configurations of system services running in a cluster of hosts by managing lifecycle of virtualization software

ABSTRACT

A method of managing configurations of a plurality of system services, including a first system service and a second system service, in each of a plurality of hosts, wherein each of the hosts is configured with a virtualization software for supporting execution of virtual machines therein includes steps of: upon receiving an application programming interface (API) call to apply configurations of the system services defined in a desired configuration file to the system services, parsing the desired configuration file to identify a first configuration for the first system service and a second configuration for the second system service, and storing the first and second configurations in accordance with a configuration schema defined for the first and second system services, wherein the first system service executes with the stored first configuration applied thereto and the second system service executes with the stored second configuration applied thereto.

A typical software stack for configuration management of a systemincludes an application programming interface (API) layer, whichprovides an endpoint to configure and monitor the system, a businesslogic layer, which contains the API implementation, and a persistencelayer, which persists any configuration or state changes in the systemonto a disk. In the typical system, configuration actions performed byan end user are not persisted while the system is live. It is thusimpossible to determine the configuration tasks previously performed bythe user, especially after a long period of time has passed sinceboot-up of the system. Rather, only the resulting state of those tasksis persisted. The system can thus only report the current configurationstate, and it is impossible to revert to a certain configuration state.In fact, it is difficult to even revert to the initial defaultconfiguration state.

The inability to determine the configuration tasks previously performedis especially problematic if the user must manage the system at a largescale. As the number of configurations that must be set and monitoredincreases, the complexity of managing the system grows. Only ad hocsolutions are available, and such solutions only provide configurationand compliance support for a limited set of configurations.

As disclosed in U.S. patent application Ser. No. 16/837,676, filed Apr.1, 2020, the entire contents of which are incorporated by referenceherein, a system may be implemented that defines which properties needto be persisted upfront in a configuration schema. The configurationschema may define such properties as either configurations or states. Aconfiguration is data that the user provides as part of a configurationaction. A state is data that the system generates internally, the statebeing further classified as either vital or cached. The system persistsconfigurations and vital states across reboots but does not persistcached states.

By defining properties using configuration schemas, configurationactions can be tracked by storing updates to configurations in adatabase. As a result, configuration changes can be easily detectedwhile the system is live. However, the system may include many services,including network time protocol (NTP) service, secure shell (SSH)service, authentication service, firewall service, network service,storage service, keyboard service, etc. It is still burdensome for theuser to manage the configurations for all these different servicesseparately.

SUMMARY

Accordingly, one or more embodiments provide a method of managingconfigurations of a plurality of system services, including a firstsystem service and a second system service, in each of a plurality ofhosts, wherein each of the hosts is configured with a virtualizationsoftware for supporting execution of virtual machines therein. Themethod includes the steps of: upon receiving an application programminginterface (API) call to apply configurations of the system servicesdefined in a desired configuration file to the system services, parsingthe desired configuration file to identify a first configuration for thefirst system service and a second configuration for the second systemservice, and storing the first and second configurations in accordancewith a configuration schema defined for the first and second systemservices, wherein the first system service executes with the storedfirst configuration applied thereto and the second system serviceexecutes with the stored second configuration applied thereto.

Further embodiments include a non-transitory computer-readable storagemedium comprising instructions that cause a computer system to carry outthe above method, as well as a computer system configured to carry outthe above method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a virtualized computing system of the priorart in which configurations are persisted in files that are stored inlocal storage units.

FIG. 2 is a block diagram of a virtualized computing system in whichconfigurations are persisted in key-value stores provisioned in localstorage units.

FIG. 3 is a block diagram of a computing system in which configurationsare persisted in key-value stores provisioned in local storage units,according to embodiments.

FIG. 4 is a flow diagram of a method carried out by a virtual machinemanagement server to generate a master configuration schema, accordingto an embodiment.

FIG. 5 is a flow diagram of a method carried out by a schema engine andvirtual machine management server to embed configuration schemas inmetadata of software installation bundles, automatically generate APIdocumentation for APIs that are called to configure system services of acomputing system, and generate a master configuration schema, accordingto an embodiment.

FIG. 6 is a flow diagram of a method carried out by a virtual machinemanagement server and host to persist configurations of a desiredconfiguration JSON file in a key-value store, according to anembodiment.

FIG. 7A is an example of a desired configuration JSON file and key-valuestore.

FIG. 7B is an example of a desired configuration JSON file and key-valuestore after set API commands are executed.

FIG. 7C is an example of a desired configuration JSON file and key-valuestore after update API commands are executed.

FIG. 7D is an example of a desired configuration JSON file and key-valuestore after delete API commands are executed.

FIG. 7E is an example of a result of a get API command.

DETAILED DESCRIPTION

FIG. 1 is a simplified block diagram of a virtualized computing systemof the prior art in which configurations are persisted in files that arestored in local storage units 30. The virtualized computing system ofFIG. 1 includes a virtual machine (VM) management server 10 that managesthe lifecycle of VMs running in a cluster of hosts 20. To configuresystem services running in hosts 20, an end user operates a userinterface (UI) (not shown) on VM management server 10 to makeconfiguration API calls to hosts 20. A host daemon 21 running in eachhost 20 receives and processes these API calls. If an API call requiresthe virtualized computing system to set a configuration for a particularsystem service, host daemon 21 instructs a configuration persistencelayer 22 in host 20 to persist the configuration in a local storage unit30. The configuration is stored as a file in local storage unit 30corresponding to the system service (i.e., file F1, file F2, . . . ,file Fn). For example, if the configuration for the system service ispersisted in file F1, then the next time a host 20 executes the systemservice, host 20 executes the system service with the configurationstored in file F1.

FIG. 2 is a simplified block diagram of a virtualized computing systemin which configurations are persisted in key-value stores 140provisioned in local storage units 130. The virtualized computing systemof FIG. 2 includes a VM management server 100 that manages the lifecycleof VMs running in a cluster of hosts 120. To configure system servicesrunning in hosts 120, an end user operates a UI (not shown) on VMmanagement server 100 to make configuration API calls to hosts 120. Ahost daemon 121 running in each host 120 receives these API calls andpasses them to a configuration store 122 for processing. Configurationstore 122 exposes configurations for different system services asconfiguration objects, each configuration object being backed by acorresponding configuration schema. Configuration store 122 records allupdates to the configurations of system services in key-value store 140.In key-value store 140, a “key” corresponds to a system service, and acorresponding “value” for the key stores one or more configurationproperties and one or more internal states for that system service.

FIG. 3 is a simplified block diagram of a computing system 300 in whichconfigurations are persisted in key-value stores 360 provisioned inlocal storage units 350, according to embodiments. Computing system 300includes a schema engine 310, an image depot 320, a VM management server330, and a cluster of hosts 340.

In computing system 300, configurations for system services are definedin schemas. Software publishers of system services define the schemas inschema definition files, e.g., VMware Managed Object Design Language 2(VMODL2) files 302. Each VMODL2 file 302 corresponds to a system service(i.e., system services 1 through n).

Schema engine 310 is a physical or virtual server that processes VMODL2files 302 and generates schemas from the VMODL2 files. In theembodiments illustrated herein, the schemas are in the format ofJavaScript Object Notation (JSON) files 304. For each VMODL2 file 302,schema engine 310 generates an individual JSON file, e.g.,SS1.schema.json, referred to herein as a “configuration schema.”Additionally, for each VMODL2 file 302 that contains a definition for adefault configuration, schema engine 310 generates a default JSON file,e.g., SS1.default.json, referred to herein as a “default schema.” Adefault schema for a system service contains the initial configurationsfor the system service, and a host 340 may revert to these initialconfigurations as described in U.S. patent application Ser. No.16/837,760, filed Apr. 1, 2020, the entire contents of which areincorporated by reference herein. In the example given in FIG. 3 ,default schemas are available for system services 1 and n, but not forsystem service 2.

Image depot 320 is a storage service that stores software installationbundles (SIBs) for system services executed on hosts 340, i.e., “SS1SIB,” “SS2 SIB,” and “SSn SIB.” Each SIB contains the binaries forexecuting a system service on a host 340. Additionally, each SIB embedsJSON files generated by schema engine 310 in its metadata. For example,SS1 SIB contains the binaries for executing system service 1 and alsoembeds SS1.schema.json and SS1.default.json in its metadata.

Hosts 340 are servers that may be constructed on server grade hardwareplatforms such as x86 architecture platforms. Each host 340 contains avirtualization software layer (not shown) supporting a VM executionspace for concurrently instantiating and executing VMs. Hosts 340 runsystem services based on configurations stored in key-value stores 360,which are persisted in local storage units 350.

Local storage units 350 are provisioned in shared storage that maycomprise, e.g., magnetic disks or flash memory in a storage area network(SAN), and a separate local storage unit 350 is provisioned for eachhost 340. Each host 340 maintains its own key-value store 360 in localstorage unit 350. In addition, each host 340 maintains a separate copyof master schema JSON file 352 and default JSON files 354.

Master schema JSON file 352 is the master configuration schema of allsystem services running in hosts 340. Each default JSON file 354 is thedefault configuration schema for one of the system services and containsthe default configuration for that system service.

Each key-value store 360 is a database in which a “key” corresponds to asystem service, and a corresponding “value” for the key stores one ormore configuration properties and one or more internal states for thatsystem service. The current configuration state of the system servicesrunning in each host 340 is maintained in key-value store 360corresponding to that host 340. “Drift” occurs when the actualconfiguration state, as persisted in key-value store 360, deviates fromthe desired configuration state, as defined in a desired configurationJSON file 336 of a local storage unit 334 accessible by VM managementserver 330. The user defines the desired configuration state in desiredconfiguration JSON file 336 using APIs 306 as described below.

VM management server 330 is a physical or virtual server that managesthe lifecycle of VMs running in hosts 340. VM management server 330 alsomanages installation and configuration of system services in hosts 340.During installation of system services, hosts 340 retrieve binaries ofthe system services from image depot 320 and load them into memory forexecution therein, and configuration manager 332 extracts theconfiguration schemas and any default schemas embedded in the metadataof these system services. Configuration manager 332 generates masterschema JSON file 352 from the configuration schemas of these systemservices and stores master schema JSON file 352 in local storage units350. In addition, configuration manager 332 stores any default schemasin local storage units 350.

Each host 340 contains a host configuration manager 342 for accessingkey-value store 360 in response to an “apply” API call received fromconfiguration manager 332. To make the apply API call, configurationmanager 332 accesses desired configuration JSON file 336 from localstorage unit 334 and transmits desired configuration JSON file 336 tohost configuration manager 342 along with the apply API call. Inresponse to the apply API call, host configuration manager 342 checksfor drift by comparing the desired configuration state expressed indesired configuration JSON file 336 with the actual configuration state,as persisted in key-value store 360. If there is drift in any of theconfiguration objects, plug-ins (not shown) in host 340 update key-valuestore 360 to apply all the configurations that are in drift.

To configure system services running in hosts 340, an end user operatesa UI (not shown) on VM management server 330 to make configuration APIcalls 306, which are exposed by configuration manager 332. ConfigurationAPI calls 306 include “set,” “update,” “delete,” and “get” API calls. Inresponse, configuration manager 332 updates desired configuration JSONfile 336 and makes an apply API call to host configuration managers 342running in hosts 340 to apply the configurations defined in the updateddesired configuration JSON file 336, as illustrated in FIGS. 7B-7D.

A set API call 306 creates or overwrites a configuration object indesired configuration JSON file 336 corresponding to the system serviceidentified in the API call, as illustrated in FIG. 7B. An update APIcall 306 updates a configuration object in desired configuration JSONfile 336 for the system service identified in the API call, asillustrated in FIG. 7C. A delete API call 306 deletes part of aconfiguration object in desired configuration JSON file 336 for thesystem service identified in the API call, as illustrated in FIG. 7D.Changes made to desired configuration JSON file 336 pursuant to set,update, and delete API calls result in changes to configuration objectsin key-value store 360 via apply API calls. A get API call 306 retrievesa configuration object from a desired configuration JSON file 336 forthe system service identified in the API call, as illustrated in FIG.7E.

FIG. 4 is a flow diagram of a method 400 carried out by VM managementserver 330 to generate an initial master schema JSON file 352, accordingto an embodiment.

At step 410, configuration manager 332 initializes a master schema JSONfile 352 without any configuration schemas. At step 412, configurationmanager 332 retrieves all the SIBs from image depot 320, each SIBcontaining a configuration schema for a system service embedded in itsmetadata.

At step 414, configuration manager 332 selects a SIB, e.g., SS1 SIB. Atstep 416, configuration manager 332 extracts the configuration schemaembedded in the selected SIB, e.g., SS1.schema.json. At step 418,configuration manager 332 adds the extracted configuration schema to themaster schema JSON file 352 initialized at step 410.

At step 420, configuration manager 332 determines if there is a SIB foranother system service to extract a configuration schema from. If thereis, then method 400 moves back to step 414. Otherwise, method 400 ends.

FIG. 5 is a flow diagram of a method 500 carried out by schema engine310 and VM management server 330 to embed configuration schemas inmetadata of SIBs, automatically generate API documentation for APIs thatare called to configure system services of computing system 300, andgenerate master schema JSON file 352, according to an embodiment.

At step 510, schema engine 310 reads VMODL2 files 302 that have beengenerated by software vendors of the system services. At step 512,schema engine 310 generates configuration schemas and default schemasfrom VMDOL2 files 302. For example, for the VMODL2 file 302 for systemservice 1, schema engine 310 generates SS1.schema.json andSS1.default.json.

At step 514, schema engine 310 embeds the configuration schemas anddefault schemas in the metadata of the SIBs of image depot 320. Forexample, schema engine 310 embeds copies of SS1.schema.json andSS1.default.json in the metadata of SS1 SIB.

At step 516, schema engine 310 filters out internal states defined inseparate copies of the configuration schemas, thus leaving onlyconfiguration properties for the associated system services. At step518, schema engine 310 generates a VMODL2 file from each filteredconfiguration schema. At step 520, schema engine 310 generates APIdocumentation from the generated VMODL2 files. Specifically, schemaengine 310 generates API documentation for set, update, delete, and getAPI calls for each system service.

At step 522, schema engine 310 transmits a notification to configurationmanager 332 that the SIBs of image depot 320 are ready for retrieval ofthe schemas.

At step 524, configuration manager 332 retrieves the SIBs from imagedepot 320. At step 526, configuration manager 332 extracts theconfiguration schemas and default schemas from the retrieved SIBs. Atstep 528, configuration manager 332 generates master schema JSON file352 from the configuration schemas extracted at step 526 according tothe method of FIG. 4 .

At step 530, configuration manager 332 stores master schema JSON file352 and the default JSON files in local storage units 350. After step530, method 500 ends.

FIG. 6 is a flow diagram of a method 600 carried out by VM managementserver 330 and a host 340 to persist configurations of desiredconfiguration JSON file 336 in a key-value store 360, according to anembodiment.

At step 610, configuration manager 332 determines if a condition forissuing an apply API call is satisfied for host 340. The condition forissuing an apply API call may be drift or an update to desiredconfiguration JSON file 336 (e.g., when a user makes one ofconfiguration API calls 306). Configuration manager 332 may periodicallytransmit a request to a host 340 to check for drift or may transmit arequest in response to a user command.

At step 612, if the condition is not satisfied, configuration manager332 returns to step 610 to check again if the condition for issuing anapply API call is satisfied. If the condition is satisfied,configuration manager 332 at step 614 transmits an apply API call tohost 340 along with desired configuration JSON file 336.

At step 616, host configuration manager 342 parses desired configurationJSON file 336 for configuration objects. At step 618, host configurationmanager 342 determines if any of the configuration objects are in drift,i.e., the actual state does not match the desired state. If not, method600 ends. If so, host configuration manager 342 at step 620 executesplug-ins associated with the configuration objects in drift to apply thedesired state and update the configuration objects in key-value store360 in accordance with master schema JSON file 352.

If any updates to the configuration objects in key-value store 360 arenot in accordance with master schema JSON file 352, host configurationmanager 342 returns an error message to configuration manager 332, andmethod 600 ends.

The updates may include a creation of a key-value entry, an update to anexisting key-value entry, or a deletion of an existing key-value entry.To create a key-value entry, a plug-in issues a “set” API command tokey-value store 360. To update an existing key-value entry, the plug-inissues an “update” API command to key-value store 360. To delete anexisting key-value entry, the plug-in issues a “delete” API command tokey-value store 360.

After step 620, method 600 ends, and host 340 runs system services withthe updated configurations specified in key-value store 360.

FIG. 7A is an example of desired configuration JSON file 336 andkey-value store 360. In the example of FIG. 7A, desired configurationJSON file 336 contains two configuration objects: one for an NTP systemservice, identified by the key “ntp,” and another for a keyboard systemservice, identified by the key “keyboard.” It should be understood thatthe example of desired configuration JSON file 336 shown in FIG. 7A issimplified for purposes of illustration and actual examples of desiredconfiguration JSON file 336 contain many more configuration objects.

Lines 710 create the NTP configuration object. As shown in lines 712,the NTP configuration object contains a “server” configuration property,and the value for the server configuration property is“time.vmware.com.” Additionally, as shown in lines 714, the NTPconfiguration object contains a “drift” vital internal state that may beset with a value of type “double.”

Lines 716 create the keyboard configuration object. As shown in lines718, no values have been set for the keyboard configuration object.However, the keyboard configuration object contains a “layout”configuration property that may be set with a value of type “string.”Additionally, the keyboard configuration object may contain one or moreinternal states (not shown).

Key-value store 360 contains an entry for an NTP configuration object.The NTP configuration object contains the value “time.vmware.com” forthe server configuration property and a value for the drift internalstate. There is no entry for a keyboard configuration object because novalues have been set for the keyboard configuration object in desiredconfiguration JSON file 336.

FIG. 7B is an example of desired configuration JSON file 336 andkey-value store 360 after a set API command is executed by configurationmanager 332 on the desired configuration JSON file 336 of FIG. 7A. Asshown in lines 720, after the set API command is executed on desiredconfiguration JSON file 336, the layout configuration property containsthe value “US Default.”

After the layout configuration property is set in desired configurationJSON file 336, configuration manager 332 issues an apply API call withdesired configuration JSON file 336 to host 340 to match the actualconfiguration state with the desired configuration state. In response,host configuration manager 342 detects that the system service“keyboard” is in drift, and issues a second set API call, represented aslines 722, to update key-value store 360 to contain an entry for akeyboard configuration object. As in desired configuration JSON file336, the keyboard configuration object contains the value “US Default”for the layout configuration property.

FIG. 7C is an example of desired configuration JSON file 336 andkey-value store 360 after an update API command is executed byconfiguration manager 332 on the desired configuration JSON file 336 ofFIG. 7B. As shown in lines 730, after the update API command is executedon desired configuration JSON file 336, the layout configurationproperty contains the value “Korean.”

After the layout configuration property is updated in desiredconfiguration JSON file 336, configuration manager 332 issues an applyAPI call with desired configuration JSON file 336 to host 340 to matchthe actual configuration state with the desired configuration state. Inresponse, host configuration manager 342 detects that the system service“keyboard” is in drift, and issues a second update API call, representedas lines 732, to update key-value store 360. The layout configurationproperty in key-value store 360 is then updated from “US Default” to“Korean.”

FIG. 7D is an example of desired configuration JSON file 336 andkey-value store 360 after a delete API command is executed byconfiguration manager 332 on the desired configuration JSON file 336 ofFIG. 7C. As shown in lines 740, after the delete API command is executedon desired configuration JSON file 336, the layout configurationproperty no longer contains a value.

After the layout configuration property is deleted from desiredconfiguration JSON file 336, configuration manager 332 issues an applyAPI call with desired configuration JSON file 336 to host 340 to matchthe actual configuration state with the desired configuration state. Inresponse, host configuration manager 342 detects that the system service“keyboard” is in drift, and issues a second delete API call, representedas lines 742, to key-value store 360. The layout configuration propertyin key-value store 360 is then deleted along with the keyboardconfiguration object.

FIG. 7E is an example of a result of a get API command executed ondesired configuration JSON file 336 of FIG. 7D. The get API commandexecuted in the example of FIG. 7E retrieves configuration propertiesand internal states for the NTP system service. The result that isreturned in response to the get API command includes the serverconfiguration property with the value time.vmware.com and the driftinternal state with the value currently stored for the drift state.

The embodiments described herein may employ various computer-implementedoperations involving data stored in computer systems. For example, theseoperations may require physical manipulation of physical quantities.Usually, though not necessarily, these quantities are electrical ormagnetic signals that can be stored, transferred, combined, compared, orotherwise manipulated. Such manipulations are often referred to in termssuch as producing, identifying, determining, or comparing. Anyoperations described herein that form part of one or more embodimentsmay be useful machine operations.

One or more embodiments of the invention also relate to a device or anapparatus for performing these operations. The apparatus may bespecially constructed for required purposes, or the apparatus may be ageneral-purpose computer selectively activated or configured by acomputer program stored in the computer. Various general-purposemachines may be used with computer programs written in accordance withthe teachings herein, or it may be more convenient to construct a morespecialized apparatus to perform the required operations.

The embodiments described herein may be practiced with other computersystem configurations including hand-held devices, microprocessorsystems, microprocessor-based or programmable consumer electronics,minicomputers, mainframe computers, etc.

One or more embodiments of the present invention may be implemented asone or more computer programs or as one or more computer program modulesembodied in computer readable media. The term computer readable mediumrefers to any data storage device that can store data that canthereafter be input into a computer system. Computer readable media maybe based on any existing or subsequently developed technology thatembodies computer programs in a manner that enables a computer to readthe programs. Examples of computer readable media are hard disk drives(HDDs), solid-state drives (SSDs), network-attached storage (NAS)systems, read-only memory (ROM), random-access memory (RAM), compactdisks (CDs), digital versatile disks (DVDs), magnetic tapes, and otheroptical and non-optical data storage devices. A computer readable mediumcan also be distributed over a network-coupled computer system so thatcomputer-readable code is stored and executed in a distributed fashion.

Although one or more embodiments of the present invention have beendescribed in some detail for clarity of understanding, certain changesmay be made within the scope of the claims. Accordingly, the describedembodiments are to be considered as illustrative and not restrictive,and the scope of the claims is not to be limited to details given hereinbut may be modified within the scope and equivalents of the claims. Inthe claims, elements and steps do not imply any particular order ofoperation unless explicitly stated in the claims.

Virtualized systems in accordance with the various embodiments may beimplemented as hosted embodiments, non-hosted embodiments, or asembodiments that blur distinctions between the two. Furthermore, variousvirtualization operations may be wholly or partially implemented inhardware. For example, a hardware implementation may employ a look-uptable for modification of storage access requests to secure non-diskdata.

Many variations, additions, and improvements are possible, regardless ofthe degree of virtualization. The virtualization software can thereforeinclude components of a host, console, or guest operating system (OS)that perform virtualization functions.

Boundaries between components, operations, and local storage units aresomewhat arbitrary, and particular operations are illustrated in thecontext of specific illustrative configurations. Other allocations offunctionality are envisioned and may fall within the scope of theinvention. In general, structures and functionalities presented asseparate components in exemplary configurations may be implemented as acombined component. Similarly, structures and functionalities presentedas a single component may be implemented as separate components. Theseand other variations, additions, and improvements may fall within thescope of the appended claims.

What is claimed is:
 1. A method of managing configurations of aplurality of system services, including a first system service and asecond system service, in each of a plurality of hosts, wherein each ofthe hosts is configured with a virtualization software for supportingexecution of virtual machines therein, said method comprising:generating a master configuration schema from a plurality of individualconfiguration schemas, each of the individual configuration schemascorresponding to one of the plurality of system services, and the masterconfiguration schema including all the individual configuration schemasand defining one or more configuration properties of the first systemservice, one or more internal states of the first system service, one ormore configuration properties of the second system service, and one ormore internal states of the second system service; and upon receiving anapplication programming interface (API) call to apply to the pluralityof system services desired configurations of the plurality of systemservices, the desired configurations of the plurality of system servicesbeing defined in a desired configuration file, parsing the desiredconfiguration file to identify a first desired configuration for thefirst system service and a second desired configuration for the secondsystem service, and storing the first and second desired configurationsin accordance with the master configuration schema, wherein the firstsystem service executes with the stored first desired configurationapplied thereto, and the second system service executes with the storedsecond desired configuration applied thereto.
 2. The method of claim 1,further comprising: generating schema definition files from filteredconfiguration schemas, wherein the filtered configuration schemas areprepared by filtering out definitions of internal states of theplurality of system services from unfiltered configuration schemas; andfor each of the plurality of system services, generating APIdocumentation for a plurality of APIs from the schema definition files,wherein the APIs for each of the plurality of system services include aset API call for defining a configuration of the system service in thedesired configuration file, an update API call for updating aconfiguration of the system service in the desired configuration file, adelete API call for deleting a configuration of the system service, anda get API call for retrieving a configuration of the system service. 3.The method of claim 1, wherein each of the plurality of system serviceshas a corresponding software installation bundle that is stored in animage depot and that contains both executable code of the system serviceand the individual configuration schema of the system service.
 4. Themethod of claim 1, further comprising: for each of the plurality ofsystem services, generating the individual configuration schema from aschema definition file prepared for the system service.
 5. The method ofclaim 4, further comprising: for each of the plurality of systemservices, generating from the schema definition file prepared for thesystem service, a default configuration schema that defines a defaultconfiguration for the system service.
 6. The method of claim 5, whereinthe first desired configuration overrides any default configurationdefined for the first system service, and the second desiredconfiguration overrides any default configuration defined for the secondsystem service.
 7. The method of claim 1, wherein the first and seconddesired configurations are stored in a key-value database, keys of thekey-value database each corresponding to one of the plurality of systemservices.
 8. A computing system comprising: a plurality of hosts eachconfigured with a virtualization software for supporting execution ofvirtual machines therein, wherein each of the hosts includes a pluralityof system services including a first system service and a second systemservice; and a management server for the cluster of hosts, wherein thecomputing system is configured to carry out a method of managingconfigurations of the plurality of system services in each of the hosts,said method comprising: generating by the management server, a masterconfiguration schema from a plurality of individual configurationschemas, each of the individual configuration schemas corresponding toone of the plurality of system services, and the master configurationschema including all the individual configuration schemas and definingone or more configuration properties of the first system service, one ormore internal states of the first system service, one or moreconfiguration properties of the second system service, and one or moreinternal states of the second system service; and upon receiving anapplication programming interface (API) call to apply to the pluralityof system services desired configurations of the plurality of systemservices, the desired configurations of the plurality of system servicesbeing defined in a desired configuration file, parsing by a host of thecluster of hosts, the desired configuration file to identify a firstdesired configuration for the first system service and a second desiredconfiguration for the second system service, and storing by the host,the first and second desired configurations in accordance with themaster configuration, wherein the first system service executes with thestored first desired configuration applied thereto, and the secondsystem service executes with the stored second desired configurationapplied thereto.
 9. The computing system of claim 8, wherein themanagement server is configured to: generate schema definition filesfrom filtered configuration schemas, wherein the filtered configurationschemas are prepared by filtering out definitions of internal states ofthe plurality of system services from unfiltered configuration schemas;and for each of the plurality of system services, generate APIdocumentation for a plurality of APIs from the schema definition files,wherein the APIs for each of the plurality of system services include aset API call for defining a configuration of the system service in thedesired configuration file, an update API call for updating aconfiguration of the system service in the desired configuration file, adelete API call for deleting a configuration of the system service, anda get API call for retrieving a configuration of the system service. 10.The computing system of claim 8, wherein each of the plurality of systemservices has a corresponding software installation bundle that is storedin an image depot and that contains both executable code of the systemservice and the individual configuration schema of the system service.11. The computing system of claim 8, further comprising: a schema engineconfigured to generate, for each of the plurality of system services,the individual configuration schema from a schema definition fileprepared for the system service.
 12. The computing system of claim 11,wherein the schema engine is further configured to generate, for each ofthe plurality of system services from the schema definition fileprepared for the system service.
 13. The computing system of claim 12,wherein the first desired configuration overrides any defaultconfiguration defined for the first system service, and the seconddesired configuration overrides any default configuration defined forthe second system service.
 14. The computing system of claim 8, whereinthe first and second desired configurations are stored in a key-valuedatabase, keys of the key-value database each corresponding to one ofthe plurality of system services.
 15. A non-transitory computer readablemedium comprising instructions to be executed in a computing system tocarry out a method of managing configurations of a plurality of systemservices, including a first system service and a second system service,in each of a plurality of hosts, wherein each of the hosts is configuredwith a virtualization software for supporting execution of virtualmachines therein, and wherein the method comprises: generating a masterconfiguration schema from a plurality of individual configurationschemas, each of the individual configuration schemas corresponding toone of the plurality of system services, and the master configurationschema including all the individual configuration schemas and definingone or more configuration properties of the first system service, one ormore internal states of the first system service, one or moreconfiguration properties of the second system service, and one or moreinternal states of the second system service; and upon receiving anapplication programming interface (API) call to apply to the pluralityof system services desired configurations of the plurality of systemservices, the desired configurations of the plurality of system servicesbeing defined in a desired configuration file, parsing the desiredconfiguration file to identify a first desired configuration for thefirst system service and a second desired configuration for the secondsystem service, and storing the first and second desired configurationsin accordance with the master configuration schema, wherein the firstsystem service executes with the stored first desired configurationapplied thereto, and the second system service executes with the storedsecond desired configuration applied thereto.